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20 February 1D73 


MEMORANDUM FOR THE RECORD: 

SUBJECT: Remarks of Mr. Andrew Marshall 


1. Mr. Marshall met with t;ie Senior Seminar from 0330 
through 1000, Thursday, 15 February to discuss the topic 
"Intelligence Products: Problems and Improvements." When we 
considered inviting Mr. Marshall originally, it was with the 
idea of hearing him talk on his work in the field of net 

assessments as a subject fitting the week's theme of "New 

Methodologies of Interest to Intelligence . " When Mr. I I S 

net with him, however, Mr. Marshall suggested that he concen- 
trate instead on the products of intelligence to which we agreed 
in view of recent changes. Accordingly, while Mr. Marshall 
occasionally referred to new methods, it was in the context of 
exhorting CIA to experiment with them, rather than the substance 
of net assessment as a new technique. 

2. Mr. Marshall began by saying that in his talk he would 
encourage us to employ ne\* techniques to a greater extent, but 
that he would also address the question of the difficulties which 
arise in trying to utilize thcia. fie then launched into a 
lengthy discussion of the intelligence product as he has per- 
ceived it from the vantage point of his earlier associations with 
the Agency and the Intelligence Community while at RAND and parti- 
cularly from the perspective of his present position as chief of 
the NSC Intelligence Committee's Net Assessment Croup. His 
remarks about the product were critical in varying degrees and 
are based, according to Mr . Marshall, on comments he has received 
from "consumers" whom he specifically identified as "members" of 
the NSC staff under Dr. Kissinger. 


STAT 


3. His criticisms are as follows: 

a. The intelligence product and the production 
organizations are deficient in their "communication . " In this 
connection he mentioned writing style. He said vie do not experi- 
ment enough with different formats. He singled out several tines 
the lack of adequate summaries to accompany longer reports and. 
critized our summaries as "not hard hitting enough," i.e., they 
do not summarize the conclusions of the longer report well or 
firmly enough. He mentioned that "uncertainties" concerning the 
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judgments and estimates in the intelligence product are not 
adequately specified for the benefit of the reader. He cited 
the ’’anonymity" of authorship as a major problem on the grounds 
that a consumer comes to trust an intelligence statement on the 
basis of knowing the individual whose judgment is expressed. 

b. The second set of criticisms concerned the "focus 
of the product." This Mr. Marshall characterized as writing 
about the right subjects in the right way. He stated that he 
recognizes that intelligence producers do not get much feedback 
--and that which is given is skewed in favor of praise. He 
assigns blame to the consumer who is busy and does not know 
what he wants, but particularly to the producer. He criticized 
the Agency for failing to do "market research." lie agreed that 
the questionnaires techniques is probably no good, but referred 
several times to the "Madison Avenue" approach. He said that we 
should be more forceful and make a greater effort to find out 
how we can help the consumer- -again pointing to personnel contact 
between the area men in the NSC staff and area experts in the 
Agency, V/ith regard to focus, he also said he believes we should 
try more experimental products, the same products in a variety of 
formats and particularly try to experiment with new methodologies. 

c. ?!r. Marshall then cited some "Organization and 
Management Changes" which he opined are necessary in order to get 
new approaches off the ground, lie said that it has proven most 
difficult to get experimental efforts going because on-line pro- 
duction offices are reluctant to put good people on problems 
being dealt with in an experimental mode. He suggested that a 
separate unit(s) be set up for the sole purpose of experimentation 
in the field of intelligence production. He cited training of 
analysts to work in the field of new methodologies as important. 

d. He then turned to the "quality of analysis." He 
stated that much of our analysis is wanting in interpretation, 
not just occasional holes, but v/ith respect to "whole programs." 

He charged that we fail to answer vital' ques tions of "Why?" or 
"What Next?” and that wc fail to put forth adequate hypotheses. 

He noted that tnere is not much in the way of reasoning behind 
our intelligence statements and that they cannot be compared 

at a future date. In sum, he claimed, we have provided few 
"basic answers" over the 10 to 20 years of our existence. In 
a provocative assertion, he claimed that in order to nrovido 
answers to fundamental questions, he could do almost as well bv 
dragging in a bunch of "people off the street." We just are not 
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good enough. As a corrective, he urged we adopt the use of 
footnotes citing sources and append annexes which lay out the 
reasoning behind intelligence statements. Me repeated that it 
is difficult to find "real analysis” in our products. I?e also 
suggested that we try harder to reach "our real readers” who, 
he suggested, are the staff persons i/orking for top leaders, 
e.g., the NSC staff. He again stressed the importance of good 
summaries for top readers. He suggested that we experiment 
more itfith visual, rather than word, presentations of material, 
e.g., cartoons. To improve the analytical quality of our pro- 
duct, he proposed that we institute an RF,D effort' in analysis, 
giving in-depth study to selected topics and that we make an 
effort to develop means of conducting a critical review on a 
sampling basis in order to get feedback. 

e. Turning to new methods, he held up Bayesian 
analysis as providing a systematic way of handling uncertainties, 
citing Dr. Kissinger’s comment to him that policy-making is 
"betting on the future.” Mr. Marshall suggested, that we might 
try expressing our intelligence judgments to Dr. Kissinger in 
the form of betting odds. He also mentioned the Bureaucratic 
Politics model of analysis followed by Graham Alison as a useful 
means of interpreting the behavior of other governments. He 
cited Kuhn's Scienti fic Revolution as useful reading, and held 
up the behavioral sciences as potentially useful in gaining 

insight into the actions of kev foreign Leaders, noting the 

STAT as helpful. 

3. As to "cautionary tales," Mr. Marshall gave his view 
that game theory has not been very useful. He cited the barriers 
that exist to innovation, giving as an example a rather lengthy 
effort on his part going back to 1953 while he was at RAND to 

get the Agency to take a new tack in analyzing Soviet weapons uro- 
grams. 

4. Mr. Marshall stated, in summary, that 3ie feels that the 
Agency should take the leadership in certain key areas vis-a-vis 
universities and think-tanks. lie said in some cases these are 
areas in which CIA is falling behind. Me urged that the Agency 
have closer contact with this outside work. He observed that we 
should not expect this work to be developed outside to. the extent 
that it then be handed to CIA and immediately applied. 
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5 The question and answer session served mainly to 
amplify’ some of Mr. Marshall's statements. One excnange con- 
cerned the unwillingness of the consumer to level with CIA 
--saying onlv good tilings to us, hut voicing dissatisfaction 
to others such as Marshall. He again implied that the burden 
of producer-consumer communication falls on the intelligence 

producer. 


STAT 


Chief, Senior Seminar 
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